Learning Directed Graphical Models from Nonlinear and Non-Gaussian Data Data Analysis Project for Master of Science in Machine Learning

Traditional constraint-based and score-based methods for learning directed graphical models from continuous data have two significant limitations: (i) they require (in practice) assuming dependencies are linear with Gaussian noise; (ii) they cannot distinguish between Markov equivalent structures. More recent structure learning methods avoid both limitations by directly exploiting characteristics of the observed data distribution resulting from nonlinear effects and non-Gaussian noise. We review these methods and focus on the additive noise model approach, which while more general than traditional approaches also suffers from two major limitations: (i) it is invertible for certain distribution families, i.e. linear Gaussians, and thus not useful for structure learning in these cases; (ii) it was originally proposed for the two variable case with a multivariate extension that requires enumerating all possible DAGs, which is ususally intractable. To address these two limitations, we introduce weakly additive noise models, which extends the additive noise model framework to cases where additive noise models are invertible and noise is not additive. We then provide an algorithm for learning equivalence classes for weakly additive noise models from data which combines a PC style search using recent advances in kernel measures of conditional dependence with greedy local searches for additive noise models. This combined approach provides a more computationally efficient search procedure for when nonlinear dependencies and/or non-Gaussian noise may be present that learns equivalence classes of structures which are often more specific than the Markov equivalence class even in the case of invertible additive noise models and non-additive noise models, addressing the limitations of both traditional structure learning methods and the additive noise model. We evaluate this approach using synthetic data and real climate teleconnection and fMRI data.